Excitation Backprop for RNNs

Supplementary Material : Top - down Neural Attention by Excitation Backprop

Learning Many Related Tasks at the Same Time with Backpropagation

Hinton [6] proposed that generalization in artificial neural nets should improve if nets learn to represent the domain's underlying regularities . Abu-Mustafa's hints work [1] shows that the outputs of a backprop net can be used as inputs through which domainspecific information can be given to the net . We extend these ideas by showing that a backprop net learning many related tasks at the sam...

2 MECHANISMS OF MULTITASK BACKPROPWe

Hinton 6] proposed that generalization in artiicial neural nets should improve if nets learn to represent the domain's underlying regularities. Abu-Mustafa's hints work 1] shows that the outputs of a backprop net can be used as inputs through which domain-speciic information can be given to the net. We extend these ideas by showing that a backprop net learning many related tasks at the same tim...

Deep Learning: Autodiff, Parameter Tying and Backprop Through Time∗

How to do parameter tying and how this relates to Backprop through time.

Kickback Cuts Backprop's Red-Tape: Biologically Plausible Credit Assignment in Neural Networks

Error backpropagation is an extremely effective algorithm for assigning credit in artificial neural networks. However, weight updates under Backprop depend on lengthy recursive computations and require separate output and error messages – features not shared by biological neurons, that are perhaps unnecessary. In this paper, we revisit Backprop and the credit assignment problem. We first decomp...